{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Movie Recommendation System with Python\n",
    "\n",
    "In this project, we'll develop a basic recommender system with Python and pandas.\n",
    "\n",
    "Movies will be suggested by similarity to other movies; this is not a robust recommendation system, but something to start out on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have two datasets:\n",
    "\n",
    "- A dataset of movie ratings.\n",
    "- A dataset of all movies titles and their ids."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Reading the ratings dataset.\n",
    "column_names = ['user_id', 'item_id', 'rating', 'timestamp']\n",
    "df = pd.read_csv('../data/u.data', sep='\\t', names=column_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>item_id</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>50</td>\n",
       "      <td>5</td>\n",
       "      <td>881250949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>172</td>\n",
       "      <td>5</td>\n",
       "      <td>881250949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>133</td>\n",
       "      <td>1</td>\n",
       "      <td>881250949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>196</td>\n",
       "      <td>242</td>\n",
       "      <td>3</td>\n",
       "      <td>881250949</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>186</td>\n",
       "      <td>302</td>\n",
       "      <td>3</td>\n",
       "      <td>891717742</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id  item_id  rating  timestamp\n",
       "0        0       50       5  881250949\n",
       "1        0      172       5  881250949\n",
       "2        0      133       1  881250949\n",
       "3      196      242       3  881250949\n",
       "4      186      302       3  891717742"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Reading the movie titles"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>item_id</th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Toy Story (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>GoldenEye (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Four Rooms (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Get Shorty (1995)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>Copycat (1995)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   item_id              title\n",
       "0        1   Toy Story (1995)\n",
       "1        2   GoldenEye (1995)\n",
       "2        3  Four Rooms (1995)\n",
       "3        4  Get Shorty (1995)\n",
       "4        5     Copycat (1995)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_titles = pd.read_csv(\"../data/Movie_Id_Titles\")\n",
    "movie_titles.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can merge them together:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>item_id</th>\n",
       "      <th>rating</th>\n",
       "      <th>timestamp</th>\n",
       "      <th>title</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>50</td>\n",
       "      <td>5</td>\n",
       "      <td>881250949</td>\n",
       "      <td>Star Wars (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>290</td>\n",
       "      <td>50</td>\n",
       "      <td>5</td>\n",
       "      <td>880473582</td>\n",
       "      <td>Star Wars (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>79</td>\n",
       "      <td>50</td>\n",
       "      <td>4</td>\n",
       "      <td>891271545</td>\n",
       "      <td>Star Wars (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>50</td>\n",
       "      <td>5</td>\n",
       "      <td>888552084</td>\n",
       "      <td>Star Wars (1977)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8</td>\n",
       "      <td>50</td>\n",
       "      <td>5</td>\n",
       "      <td>879362124</td>\n",
       "      <td>Star Wars (1977)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id  item_id  rating  timestamp             title\n",
       "0        0       50       5  881250949  Star Wars (1977)\n",
       "1      290       50       5  880473582  Star Wars (1977)\n",
       "2       79       50       4  891271545  Star Wars (1977)\n",
       "3        2       50       5  888552084  Star Wars (1977)\n",
       "4        8       50       5  879362124  Star Wars (1977)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.merge(df,movie_titles,on='item_id')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploratory Analysis\n",
    "\n",
    "Let's explore the data a bit and get a look at some of the best rated movies.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "sns.set_style('white')\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a ratings dataframe with average rating and number of ratings:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "title\n",
       "They Made Me a Criminal (1939)                5.0\n",
       "Marlene Dietrich: Shadow and Light (1996)     5.0\n",
       "Saint of Fort Washington, The (1993)          5.0\n",
       "Someone Else's America (1995)                 5.0\n",
       "Star Kid (1997)                               5.0\n",
       "Name: rating, dtype: float64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('title')['rating'].mean().sort_values(ascending=False).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "title\n",
       "Star Wars (1977)             584\n",
       "Contact (1997)               509\n",
       "Fargo (1996)                 508\n",
       "Return of the Jedi (1983)    507\n",
       "Liar Liar (1997)             485\n",
       "Name: rating, dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby('title')['rating'].count().sort_values(ascending=False).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>'Til There Was You (1997)</th>\n",
       "      <td>2.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1-900 (1994)</th>\n",
       "      <td>2.600000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101 Dalmatians (1996)</th>\n",
       "      <td>2.908257</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12 Angry Men (1957)</th>\n",
       "      <td>4.344000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>187 (1997)</th>\n",
       "      <td>3.024390</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             rating\n",
       "title                              \n",
       "'Til There Was You (1997)  2.333333\n",
       "1-900 (1994)               2.600000\n",
       "101 Dalmatians (1996)      2.908257\n",
       "12 Angry Men (1957)        4.344000\n",
       "187 (1997)                 3.024390"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings = pd.DataFrame(df.groupby('title')['rating'].mean())\n",
    "ratings.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Setting the number of ratings column:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rating</th>\n",
       "      <th>num of ratings</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>'Til There Was You (1997)</th>\n",
       "      <td>2.333333</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1-900 (1994)</th>\n",
       "      <td>2.600000</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101 Dalmatians (1996)</th>\n",
       "      <td>2.908257</td>\n",
       "      <td>109</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12 Angry Men (1957)</th>\n",
       "      <td>4.344000</td>\n",
       "      <td>125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>187 (1997)</th>\n",
       "      <td>3.024390</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             rating  num of ratings\n",
       "title                                              \n",
       "'Til There Was You (1997)  2.333333               9\n",
       "1-900 (1994)               2.600000               5\n",
       "101 Dalmatians (1996)      2.908257             109\n",
       "12 Angry Men (1957)        4.344000             125\n",
       "187 (1997)                 3.024390              41"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings['num of ratings'] = pd.DataFrame(df.groupby('title')['rating'].count())\n",
    "ratings.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visualizing the number of ratings:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlQAAAD3CAYAAADbj8pAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAASxUlEQVR4nO3dbYyddZnH8e9M20PFtERjMSvLgw+7V+YVCm6K8tC+KFuwq3XdhxDjIzHGpC9sYqJgSqhRE90gia4YCYpVF99YwKyQShNXsSBaHxPR40Vgwb7wTduktqLHaaezL87ddaydmTPzv//TM6ffT0Jyn//9MNdcHDi/+d/3ue+x6elpJEmStHjjZ7sASZKk5c5AJUmSVMhAJUmSVMhAJUmSVMhAJUmSVGjl2fzh69evn77ooouqHX9ycpJOp1Pt+Ocye1uPva3H3tZjb+uxt/UstLe//OUvD2XmujOtO6uB6qKLLuKBBx6odvxut8vExES145/L7G099rYee1uPva3H3taz0N5GxG9mW+cpP0mSpEIGKkmSpEIGKkmSpEIGKkmSpEIGKkmSpEIGKkmSpEIGKkmSpEIGKkmSpEIGKkmSpEIjH6h6x6eWZB9JknTuOquPnlkKq1et4LJbHl7QPs99YkulaiRJ0iga+RkqSZKk2gxUkiRJhQxUkiRJhQxUkiRJhQxUkiRJhQxUkiRJhQxUkiRJhQxUkiRJhQxUkiRJhQxUkiRJhQxUkiRJhQxUkiRJhQxUkiRJhQxUkiRJhVYOslFE3Aq8CegAnwMeBXYB08CTwLbMPBkRtwNbgBPA9szcX6NoSZKkYTLvDFVEbAReD1wNbAAuBu4EdmTmtcAYsDUirmjWrwduAu6qVLMkSdJQGeSU32bgF8CDwDeBh4Ar6c9SAewBNgHXAHszczozDwArI2Jd+yVLkiQNl0FO+b0EuBT4J+DlwH8D45k53aw/BlwArAUOz9jv1PjB2Q48OTlJt9tdRNmD6fV6i963Zl2joNfr2aNK7G099rYee1uPva2nzd4OEqgOA7/OzEkgI6JH/7TfKWuAI8DRZvn08Vl1Oh0mJiYWUu+ClDSpZl2joNvt2qNK7G099rYee1uPva2nzd4OcsrvMeCGiBiLiJcBLwS+3VxbBXAjsA94HNgcEeMRcQn9WaxDrVQpSZI0xOadocrMhyLiOmA//QC2DXgWuCciOkAX2J2ZUxGxD3hixnaSJEkjb6DbJmTmB88wvOEM2+0EdpaVJEmStLx4Y09JkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCKwfZKCJ+ChxtXj4L3A18GjgB7M3Mj0TEOPA54HLgT8B7MvPp9kuWJEkaLvMGqohYDYxl5sYZYz8H/gX4X+DhiHgN8HJgdWa+LiKuAj4FbK1RtCRJ0jAZZIbqcuD8iNjbbL8TOC8znwGIiEeATcDfAN8CyMwfRMRr5zvw5OQk3W53kaXPr9frLXrfmnWNgl6vZ48qsbf12Nt67G099raeNns7SKD6A3AH8AXg74A9wJEZ648BrwDWAr+bMT4VESsz88RsB+50OkxMTCy05oGVNKlmXaOg2+3ao0rsbT32th57W4+9rafN3g4SqJ4Cns7MaeCpiPgd8OIZ69fQD1jnN8unjM8VpiRJkkbFIN/yu5n+9VBExMvoB6fnI+KVETEGbAb2AY8Db2i2uwr4RZWKJUmShswgM1RfBHZFxGPANP2AdRK4D1hB/1t+P4yIHwHXR8T3gTHg3ZVqliRJGirzBqrMnATeeoZVV5223UngfS3VJUmStGx4Y09JkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCKwfZKCIuBH4CXA+cAHYB08CTwLbMPBkRtwNbmvXbM3N/lYolSZKGzLwzVBGxCrgb+GMzdCewIzOvBcaArRFxBbABWA/cBNxVp1xJkqThM8gpvzuAzwO/bV5fCTzaLO8BNgHXAHszczozDwArI2Jd28VKkiQNozlP+UXEu4CDmflIRNzaDI9l5nSzfAy4AFgLHJ6x66nxg3Mdf3Jykm63u5i6B9Lr9Ra9b826RkGv17NHldjbeuxtPfa2HntbT5u9ne8aqpuB6YjYBLwa+Apw4Yz1a4AjwNFm+fTxOXU6HSYmJgavdoFKmlSzrlHQ7XbtUSX2th57W4+9rcfe1tNmb+c85ZeZ12XmhszcCPwceAewJyI2NpvcCOwDHgc2R8R4RFwCjGfmoVYqlCRJGnIDfcvvNB8A7omIDtAFdmfmVETsA56gH9K2tVijJEnSUBs4UDWzVKdsOMP6ncDO4ookSZKWGW/sKUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVGjlfBtExArgHiCAaeB9QA/Y1bx+EtiWmScj4nZgC3AC2J6Z+yvVLUmSNDQGmaF6I0BmXg3sAD4O3AnsyMxrgTFga0RcAWwA1gM3AXdVqViSJGnIzBuoMvMbwHubl5cCR4ArgUebsT3AJuAaYG9mTmfmAWBlRKxru2BJkqRhM+8pP4DMPBERXwb+GfhX4PrMnG5WHwMuANYCh2fsdmr84GzHnZycpNvtLqbugfR6vUXvW7OuUdDr9exRJfa2Hntbj72tx97W02ZvBwpUAJn5zoj4EPBD4AUzVq2hP2t1tFk+fXxWnU6HiYmJQUtYsJIm1axrFHS7XXtUib2tx97WY2/rsbf1tNnbeU/5RcTbI+LW5uUfgJPAjyNiYzN2I7APeBzYHBHjEXEJMJ6Zh1qpUpIkaYgNMkP1APCliPgesArYDnSBeyKi0yzvzsypiNgHPEE/qG2rU7IkSdJwmTdQZebzwL+fYdWGM2y7E9hZXJUkSdIy4o09JUmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmozqB3fGpJ9pEkSaNh5VwrI2IVcC9wGXAe8DHgV8AuYBp4EtiWmScj4nZgC3AC2J6Z++uVXdfqVSu47JaHF7TPc5/YUqkaSZI07OaboXobcDgzrwVuAD4L3AnsaMbGgK0RcQWwAVgP3ATcVa9kSZKk4TLnDBXwdWB3szxGf/bpSuDRZmwP8I9AAnszcxo4EBErI2JdZh6c6+CTk5N0u91FFz+fXq9X7dhnUvN3GTa9Xu+c+n2Xkr2tx97WY2/rsbf1tNnbOQNVZv4eICLW0A9WO4A7muAEcAy4AFgLHJ6x66nxOQNVp9NhYmJicZUPYKnfgDV/l2HT7XbPqd93KdnbeuxtPfa2HntbT5u9nfei9Ii4GPgO8NXM/BpwcsbqNcAR4GizfPq4JEnSyJszUEXES4G9wIcy895m+GcRsbFZvhHYBzwObI6I8Yi4BBjPzEOVapYkSRoq811D9WHgRcBtEXFbM/Z+4DMR0QG6wO7MnIqIfcAT9EPatloFS5IkDZv5rqF6P/0AdboNZ9h2J7CzlaokSZKWEW/sKUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhAJUmSVMhA1ZLe8amq20uSpOG18mwXMCpWr1rBZbc8PPD2z31iS8VqJEnSUnKGSpIkqZCBSpIkqZCBSpIkqZCBSpIkqZCBSpIkqZCBSpIkqZCBSpIkqZCBSpIkqZCBSpIkqdBAd0qPiPXAJzNzY0S8CtgFTANPAtsy82RE3A5sAU4A2zNzf6WaR0Lv+BSrV62ovo8kSapv3kAVER8E3g483wzdCezIzO9GxOeBrRHxG2ADsB64GLgf+Ic6JY+GhT6qBnxcjSRJw2qQU37PAG+Z8fpK4NFmeQ+wCbgG2JuZ05l5AFgZEetarVSSJGlIzTtDlZn3R8RlM4bGMnO6WT4GXACsBQ7P2ObU+MG5jj05OUm3211QwQvR6/WqHftsqdmvhej1ekNTy6ixt/XY23rsbT32tp42ezvQNVSnOTljeQ1wBDjaLJ8+PqdOp8PExMQiShjMqL0Be8enFtyvWtdddbvdqv/uzmX2th57W4+9rcfe1tNmbxcTqH4WERsz87vAjcB3gKeB/4iIO4C/BcYz81ArFer/ed2VJEnDaTGB6gPAPRHRAbrA7sycioh9wBP0r8va1mKNkiRJQ22gQJWZzwFXNctP0f9G3+nb7AR2tleaJEnS8uCNPSVJkgoZqCRJkgoZqCRJkgoZqCRJkgoZqCRJkgoZqCRJkgoZqEZc7/hU1e0lSdLibuypZWShd1f/9UdvGGi7mbfqr/V4G0mSlgsDlf6Cj7eRJGnhPOUnSZJUyECls8JruyRJo8RTfjorFnpq0dOKkqRh5gyVJElSIQOVJElSIQOVJElSIQOVio3SBeNeLC9JWgwvSlexUbp3lRfLS5IWwxkqjSxnjyRJS8UZKi0Li3m8zSjNnEmShpuBSsvCsIajxQQ9n30oSaPHQCUVWKqgt9AQZmiTpKVloJKW2HxhZ2Ji4q/GFhrcfv3RG1qvS5I0OwOVtMSWYlZrMT9jKULYUoW22X7OmcLqfPtI0iBaDVQRMQ58Drgc+BPwnsx8us2fIamOxQa9YZw5G9Zr7iSNrrZnqN4MrM7M10XEVcCngK0t/wxJy9RSzZxJ0lJrO1BdA3wLIDN/EBGvbfn4ks4xo3Th/1LsY12jV9dcp6pL6xpWw3o5wVzGpqenWztYRHwBuD8z9zSvDwCvyMwTs2x/EPhNawVIkiTVc2lmrjvTirZnqI4Ca2a8Hp8tTAHMVpQkSdJy0vajZx4H3gDQXEP1i5aPL0mSNHTanqF6ELg+Ir4PjAHvbvn4kiRJQ6fVa6gkSZLORW2f8pMkSTrnGKgkSZIKGagkSZIKjeSz/HwETnsiYj3wyczcGBGvAnYB08CTwLbMPBkRtwNbgBPA9szcf9YKXgYiYhVwL3AZcB7wMeBX2NtiEbECuAcI+r18H9DD3rYmIi4EfgJcT793u7C3rYiIn9K//RDAs8DdwKfp93FvZn7Ez7eFi4hbgTcBHfq9e5QK79tRnaF6M80jcIBb6D8CRwsUER8EvgCsbobuBHZk5rX0v8W5NSKuADYA64GbgLvORq3LzNuAw00fbwA+i71tyxsBMvNqYAfwcexta5o/Bu4G/tgM2duWRMRqYCwzNzb/vBv4PPBW+k8hWR8Rr8HPtwWJiI3A64Gr6b8vL6bS+3ZUA9VfPAIH8BE4i/MM8JYZr6+kn+wB9gCb6Pd6b2ZOZ+YBYGVEeMPWuX0duK1ZHqP/15C9bUFmfgN4b/PyUuAI9rZNd9D/kP9t89retudy4PyI2BsR/xMR1wHnZeYzmTkNPMKf++vn2+A2078n5oPAN4GHqPS+HdVAtRb43YzXUxExkqc3a8rM+4HjM4bGmv+wAY4BF/DXvT41rllk5u8z81hErAF2059JsbctycwTEfFl4D+B+7C3rYiIdwEHM/ORGcP2tj1/oB9YN9M/Vf2lZuyU2frr59vcXkI/dP4b/b7eR/8pLq2/b0c1UC3oETga2MkZy2vo//V/eq9PjWsOEXEx8B3gq5n5NextqzLzncDf07+e6gUzVtnbxbuZ/o2bvwu8GvgKcOGM9fa2zFPAfzUzJE/R/3B/8Yz1s/XXz7e5HQYeyczJzEz611TODEqtvW9HNVD5CJw6ftacjwa4EdhHv9ebI2I8Ii6h/x/3obNV4HIQES8F9gIfysx7m2F724KIeHtzASr0/7o/CfzY3pbLzOsyc0NmbgR+DrwD2GNvW3MzzfVQEfEy4Hzg+Yh4ZUSM0Z+5OtVfP98G9xhwQ0SMNX19IfDtGu/bUZ0m9BE4dXwAuCciOkAX2J2ZUxGxD3iCfkDfdjYLXCY+DLwIuC0iTl1L9X7gM/a22APAlyLie8AqYDv9fvq+rcP/J7Tni8CuiHiM/rfPbqb/B8F9wAr61/f8MCJ+hJ9vA8vMh5rr0fbz5/fjs1R43/roGUmSpEKjespPkiRpyRioJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSChmoJEmSCv0fGlbkiqQLTBEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10,4))\n",
    "ratings['num of ratings'].hist(bins=40)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<AxesSubplot:>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlQAAAD5CAYAAADhhaswAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAVMklEQVR4nO3dfbBcdX3H8fclyeYiDRmtohXFlKn9zrZTHwo1VATSFgYRLC1/tI4jVK1Up1gfZxAEq9OCQseixdZqoxS1Oo4CdhCkan2gAXWoVKfGLl8HanRaHwbRhGDd3CTc/rGbzjYm9949v32657xfM5nZPWf37Pd7fzd7P/d3fvfs3OLiIpIkSaruiGkXIEmStNoZqCRJkgoZqCRJkgoZqCRJkgoZqCRJkgoZqCRJkgqtXcmDImIzcHVmbomIY4CtwCOBNcAFmXlfRFwIvBTYB1yRmbeMq2hJkqRZMrfcdagi4mLgfODHmXlSRFwPfCIzPxIRvwE8Argb+DRwIjAP3AGcmJl7ljr25s2bF4899tjyLpawsLBAq9Ua62vMsib33+Teodn923sze4dm99/k3mEy/X/961//QWY+5lD7VjJDdR9wHvCB/v2TgX+PiH8GdgCvBH4LuLMfoPZExL3AU4B/XerAxx57LDfddNOKmqiq0+nQbrfH+hqzrMn9N7l3aHb/9t7M3qHZ/Te5d5hM/xHxrcPtW3YNVWbeCOwd2LQJ+FFmng58G3gdcDSwa+Axu4GNVYqVJElabVa0huogDwA3929/HLgS+DKwYeAxG4Cdyx1oYWGBTqdToYSV63a7Y3+NWdbk/pvcOzS7f3tvZu/Q7P6b3DtMv/8qgeoO4Dn0TgGeCnwduAu4MiLmgfVAG9i+3IFardbYp+ecAm1u/03uHZrdv703s3dodv9N7h2m33+Vyya8FrggIr4APBt4c2Z+D7gW2AZ8FrgsM7ujK1OSJGl2rWiGKjN3ACf1b38LOOMQj9lK73IKkiRJjeKFPSVJkgoZqCRJkgoZqCRJkgoZqCRJkgoZqCTVznGbjh/6Od29+8dQiaSmqHIdKkmaaUcduZ5Nl9w61HN2XHX2mKqR1ATOUEmSJBUyUEmSJBUyUEmSJBUyUEmSJBUyUEmSJBUyUEmSJBUyUEmSJBUyUEmSJBUyUEmSJBUyUEmSJBUyUEmSJBUyUEmSJBUyUEmSJBVaUaCKiM0R8fmDtj0/Ir44cP/CiPhyRHwpIs4ZcZ2SJEkza9lAFREXA+8B5ge2PR34Q2Cuf/9xwCuAk4EzgbdExPpxFCxJkjRrVjJDdR9w3oE7EfGzwJuBVw085hnAnZm5JzN3AfcCTxlhnZIkSTNr2UCVmTcCewEiYg3wXuA1wO6Bhx0N7Bq4vxvYOLoyJUmSZtfaIR9/AvBk4G/pnQL8pYh4O/BZYMPA4zYAO5c72MLCAp1OZ8gShtPtdsf+GrOsyf03uXdodv/tdrvS8+rw9WryuEOz+29y7zD9/ocKVJl5F/DLABGxCfhwZr6qv4bqyoiYB9YDbWD7csdrtVqV3/hWqtPpjP01ZlmT+29y72D/VdTh69X0cW9y/03uHabf/0gum5CZ3wOuBbbRm626LDO7ozi2JEnSrFvRDFVm7gBOWmpbZm4Fto6wNkmSpFXBC3tKkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVMlBJkiQVWruSB0XEZuDqzNwSEU8D3gHsB/YAF2Tm9yPiQuClwD7gisy8ZUw1S5IkzZRlZ6gi4mLgPcB8f9NfAX+SmVuAm4DXRcTjgFcAJwNnAm+JiPVjqViSJGnGrOSU333AeQP3n5eZX+3fXgt0gWcAd2bmnszcBdwLPGWUhUqSJM2qZU/5ZeaNEbFp4P53ASLimcDLgVPpzUrtGnjabmDjcsdeWFig0+kMWfJwut3u2F9jljW5/yb3Ds3uv91uV3peHb5eTR53aHb/Te4dpt//itZQHSwifh+4DDg7M++PiAeBDQMP2QDsXO44rVar8hvfSnU6nbG/xixrcv9N7h3sv4o6fL2aPu5N7r/JvcP0+x86UEXEC+gtPt+SmT/sb74LuDIi5oH1QBvYPrIqJUmSZthQgSoi1gDXAt8GbooIgNsz840RcS2wjd66rMsyszvqYiVJkmbRigJVZu4ATurffdRhHrMV2DqasiRJklYPL+wpSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUyEAlSZJUaO1KHhQRm4GrM3NLRPwCcD2wCGwHLsrMhyPijcDZwD7gVZl515hqliRJminLzlBFxMXAe4D5/qZrgMsz8xRgDjg3In4VOA3YDDwP+JvxlCtJkjR7VnLK7z7gvIH7JwC392/fBpwOPAv4VGYuZua3gbUR8ZiRVipJkjSjlj3ll5k3RsSmgU1zmbnYv70b2AgcDTww8JgD2+9f6tgLCwt0Op2hCh5Wt9sd+2vMsib33+Teodn9t9vtSs+rw9eryeMOze6/yb3D9Ptf0Rqqgzw8cHsDsBN4sH/74O1LarVald/4VqrT6Yz9NWZZk/tvcu9g/1XU4evV9HFvcv9N7h2m33+Vv/L7SkRs6d8+C9gG3AmcGRFHRMRxwBGZ+YMR1ShJkjTTqsxQvRbYGhEtoAPckJn7I2Ib8EV6Ie2iEdYoSZI001YUqDJzB3BS//Y36P1F38GPeRPwptGVJkmStDp4YU9JkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJqqC7d/9EniNpdahyHSpJarz5dWvYdMmtQz1nx1Vnj6kaSdPmDJUkSVIhA5UkSVIhA5UkNZzrwaRyrqGSJHoBYX7dmmmXMRWuB5PKGagkieFDhYFC0iBP+UmSJBUyUEmSJBUyUElSjRy36fhplyA1kmuoJKlGjjpyvQvMpSlwhkqSJKmQgUqSJKmQgUqSJKlQpTVUEbEOeB+wCdgPXAjsA64HFoHtwEWZ+fBIqpQkSZphVWeongOszcxnAn8GXAlcA1yemacAc8C5oylRkiRptlUNVN8A1kbEEcDRwF7gBOD2/v7bgNPLy5Ok+vAz86T6qnrZhIfone67B3g0cA5wamYu9vfvBjYud5CFhQU6nU7FElam2+2O/TVmWZP7b3Lv0Oz+2+32tEs4pKqfmTerH4kzi99fTf6+b3LvMP3+qwaqVwOfzMxLI+KJwGeB1sD+DcDO5Q7SarXG/sbX6XRm9s11Eprcf5N7B/vX+M3i91eTv++b3DtMv/+qp/x+BOzq3/4hsA74SkRs6W87C9hWVpokScMZ9krxnlLVqFSdoXobcF1EbKM3M/V64MvA1ohoAR3ghtGUKEnSygx7pXivEq9RqRSoMvMh4PcOseu0snIkSZJWHy/sKUmSVMhAJUka2rBrj1yrpLqruoZKktRgw14CwrVKqjtnqCRJkgoZqCRJkgoZqCRJkgoZqCRNlIuZJdWRi9IlTZSLmSXVkTNUkiRJhQxUkiRJhWofqIb9oExwzYYkSRpO7ddQDftBmeCaDUmSNJzaz1BJkiSNm4FKkiSpkIFKkjR2Vdamup5Vq0nt11BJTTXsH2R09+5nft2aMVWjphv2+mPgelatLgYqqaaG/YMMf3hJUnWe8pMkSSpkoJIEuMZFkkpUPuUXEZcCvw20gHcCtwPXA4vAduCizHx4BDVKmoCqa1xcFyNJFWeoImIL8EzgZOA04InANcDlmXkKMAecO6IaJUmSZlrVU35nAl8DPgZ8HLgFOIHeLBXAbcDpxdVJkiStAlVP+T0aeBJwDvDzwM3AEZm52N+/G9i43EEWFhbodDoVS1iZdrtd6XnjrmtSut1ubXoZVpN7h+rf+7No2HGsU+9NN4mxr8v7RNPf86bdf9VA9QBwT2YuABkRXXqn/Q7YAOxc7iCtVmtm3/hmta5hdTqd2vQyrCb3XjeOY3NNYuzr8v3V9Pe8afdf9ZTfHcCzI2IuIh4PHAV8pr+2CuAsYNsI6pMkSZp5lWaoMvOWiDgVuIteKLsI+CawNSJaQAe4YWRVSpIkzbDKl03IzIsPsfm0glokSZJWJS/sKUmSVMhAJUmSVMhApdo6btPxQz/Hj1KRZof/H7WaVF5DJc26o45c78eiSKtY1Y9DkqbBGSpJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCa0ueHBHHAHcDZwD7gOuBRWA7cFFmPlxaoCRJ0qyrPEMVEeuAdwM/6W+6Brg8M08B5oBzy8uTJEmafSWn/N4KvAv4Tv/+CcDt/du3AacXHFvSgO7e/dMuQZK0hEqn/CLihcD9mfnJiLi0v3kuMxf7t3cDG5c7zsLCAp1Op0oJK9Zutys9b9x1TUq3261NL8Oq09i32202XXLrUM/ZcdXZY6pm8oYdk6pjr+bp7t3P/Lo1Qz3nxz/Zw7d3/OeYKqquye/3MP3+q66hejGwGBGnA08D3g8cM7B/A7BzuYO0Wq2ZfeOb1bqG1el0atPLpPj1mj2OicZlft2aSr+szOL3ZNPf76fdf6VTfpl5amaelplbgK8CFwC3RcSW/kPOAraNokBJkqRZV/RXfgd5LbA1IlpAB7hhhMeWJEmaWcWBqj9LdcBppceTVpMq6y+qPEeSNNtGOUMlNU7V9ReSpHrxSumSJEmFDFTShHlNqeH49ZK0GnjKT5owTxMOx6+XpNXAGSpJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBipJkqRCBippgJ8bJ2k5Vd4nfG+pPz/LTxow7OfG+ZlxUvP4+ZI6FGeoJEmSChmoJEmSClU65RcR64DrgE3AeuAK4D+A64FFYDtwUWY+PJIqJUmSZljVGaoXAA9k5inAs4G/Bq4BLu9vmwPOHU2JkiRJs61qoPoo8Ib+7TlgH3ACcHt/223A6WWlSZIkrQ6VTvll5kMAEbEBuAG4HHhrZi72H7Ib2DiSCiVJkmZc5csmRMQTgY8B78zMD0XEXwzs3gDsXO4YCwsLdDqdqiWsSLvdrvS8cdc1Kd1utza9DKvq2EvSOIz7vbjJ7/cw/f6rLkp/LPAp4OWZ+Zn+5q9ExJbM/DxwFvC55Y7TarVm9oferNY1rE6nU5teJGk1G/d7cdPf76fdf9UZqtcDjwTeEBEH1lK9Erg2IlpAh96pQEmSpNqruobqlfQC1MFOKytHkiRp9fHCnpIkSYUMVJIkSYUMVJIkSYUMVJqK7t79Y328JEmTVPk6VFKJ+XVr2HTJrSt+/I6rzh5jNZIklXGGSpIkqZCBSpIkqZCBquaO23T8UI93rZIkScNzDVXNHXXk+qHWKt3z588e+jW6e/czv27N0M+TJKkuDFT6f4ZdLA4uGJckjVaVX9SHPSMzagYqSZI0U1bjL/euoZIkSSpkoJIkSSpkoNKq4F8fSlrN/HSI+nMNlVaF1Xg+XZIO8NMh6s8ZKhXzNylJGq0q76vT/iu3pnOGSsWcPZKk0fJ9dfVxhmpKqvz24UyQJOlw/LkyXc5QTYm/fUiSRsmfK9M10kAVEUcA7wSeCuwBXpKZ947yNSRJkmbNqE/5/Q4wn5m/DlwC/OWIjz+znDaVJKm5Rn3K71nAPwFk5pci4sQRH38iqnyG0CT+JNYPIZYkjVKVnyv+LDq0UQeqo4FdA/f3R8TazNw34tcZq1k9Dz2rdUmSVid/rozO3OLi4sgOFhHXAF/KzI/07/9XZj5hicffD3xrZAVIkiSNz5My8zGH2jHqGao7gecCH4mIk4CvLfXgwxUlSZK0mow6UH0MOCMivgDMAS8a8fElSZJmzkhP+UmSJDWRV0qXJEkqZKCSJEkqZKCSJEkqVLvP8ouIzcDVmbnloO3PBf4U2Adcl5lbp1DeWC3R+6uBlwD39ze9NDNzwuWNRUSsA64DNgHrgSsy8+aB/bUe9xX0X+exXwNsBQJYBF6WmdsH9td97Jfrv7Zjf0BEHAPcDZyRmfcMbK/12MOSvTdh3P8NeLB/95uZ+aKBfRcCL6U39ldk5i2TqqtWgSoiLgbOB3580PZ1wNuAX+vvuzMibs7M70++yvE4XO99JwAXZObdk61qIl4APJCZ50fEo4CvAjdDM8adJfrvq/PYPxcgM0+OiC3AlcC50JixP2z/fXUe+wNj/G7gJ4fYXuuxP1zvfXUf93lg7uCJg/6+xwGvAE4E5oE7IuLTmblnErXV7ZTffcB5h9jeBu7NzB9l5gJwB3DqRCsbv8P1Dr3/YJdGxB0RcekEa5qEjwJv6N+eo/dbyQFNGPel+ocaj31m/iPwR/27TwJ2Duyu/dgv0z/UeOz73gq8C/jOQdtrP/Ycvneo/7g/FXhERHwqIj7bv+blAc8A7szMPZm5C7gXeMqkCqtVoMrMG4G9h9h18Efi7AY2TqSoCVmid4APAy8DfhN4VkScM7HCxiwzH8rM3RGxAbgBuHxgdxPGfan+ocZjD5CZ+yLifcA7gA8O7Kr92MOS/UONxz4iXgjcn5mfPMTuWo/9Mr1Djce973/oBcoz6fX5wYg4cLZtqmNfq0C1hAeBDQP3N/DTv83VUkTMAW/PzB/0f1u7FXj6lMsaqYh4IvA54AOZ+aGBXY0Y98P134SxB8jMPwB+EdgaEUf1Nzdi7OHQ/Tdg7F9M7yLSnweeBry/f7oH6j/2h+29AeMO8A3gHzJzMTO/ATwA/Fx/31THvlZrqJbQAZ7cX2PyEL3p37dOt6SJORrYHhFteusJfpPeIuZaiIjHAp8CXp6Znzlod+3HfZn+6z725wNPyMy30Put9eH+P2jG2C/Vf63HPjP/7xReP1i8LDO/199U67Ffpvdaj3vfi4FfAf44Ih5Pr+fv9vfdBVzZX2e1nt7p3+2HPMoY1DpQRcTzgZ/JzL+LiNcAn6Q3K3ddZv73dKsbr4N6fz29GYw9wGcy8xPTrW6kXg88EnhDRBxYS7QVOKoh475c/3Ue+5uAv4+IfwHWAa8CfjcimvJ/frn+6zz2P8X3+0a83wO8F7g+Iu6g99etLwZeERH3ZubNEXEtsI3e2F+Wmd1JFeZHz0iSJBVqyhoqSZKksTFQSZIkFTJQSZIkFTJQSZIkFTJQSZIkFTJQSZIkFTJQSZIkFTJQSZIkFfpf2HT7SwumezAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 720x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(10,4))\n",
    "ratings['rating'].hist(bins=40)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It makes intuitive sense for most ratings to be around the 3.0 mark."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Recommending Similar Movies"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step is to create a martix that has the user ids on one axis and the movie titles on another. Each cell will then consist of the rating of a movie by a particular user"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>title</th>\n",
       "      <th>'Til There Was You (1997)</th>\n",
       "      <th>1-900 (1994)</th>\n",
       "      <th>101 Dalmatians (1996)</th>\n",
       "      <th>12 Angry Men (1957)</th>\n",
       "      <th>187 (1997)</th>\n",
       "      <th>2 Days in the Valley (1996)</th>\n",
       "      <th>20,000 Leagues Under the Sea (1954)</th>\n",
       "      <th>2001: A Space Odyssey (1968)</th>\n",
       "      <th>3 Ninjas: High Noon At Mega Mountain (1998)</th>\n",
       "      <th>39 Steps, The (1935)</th>\n",
       "      <th>...</th>\n",
       "      <th>Yankee Zulu (1994)</th>\n",
       "      <th>Year of the Horse (1997)</th>\n",
       "      <th>You So Crazy (1994)</th>\n",
       "      <th>Young Frankenstein (1974)</th>\n",
       "      <th>Young Guns (1988)</th>\n",
       "      <th>Young Guns II (1990)</th>\n",
       "      <th>Young Poisoner's Handbook, The (1995)</th>\n",
       "      <th>Zeus and Roxanne (1997)</th>\n",
       "      <th>unknown</th>\n",
       "      <th>Á köldum klaka (Cold Fever) (1994)</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>user_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 1664 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "title    'Til There Was You (1997)  1-900 (1994)  101 Dalmatians (1996)  \\\n",
       "user_id                                                                   \n",
       "0                              NaN           NaN                    NaN   \n",
       "1                              NaN           NaN                    2.0   \n",
       "2                              NaN           NaN                    NaN   \n",
       "3                              NaN           NaN                    NaN   \n",
       "4                              NaN           NaN                    NaN   \n",
       "\n",
       "title    12 Angry Men (1957)  187 (1997)  2 Days in the Valley (1996)  \\\n",
       "user_id                                                                 \n",
       "0                        NaN         NaN                          NaN   \n",
       "1                        5.0         NaN                          NaN   \n",
       "2                        NaN         NaN                          NaN   \n",
       "3                        NaN         2.0                          NaN   \n",
       "4                        NaN         NaN                          NaN   \n",
       "\n",
       "title    20,000 Leagues Under the Sea (1954)  2001: A Space Odyssey (1968)  \\\n",
       "user_id                                                                      \n",
       "0                                        NaN                           NaN   \n",
       "1                                        3.0                           4.0   \n",
       "2                                        NaN                           NaN   \n",
       "3                                        NaN                           NaN   \n",
       "4                                        NaN                           NaN   \n",
       "\n",
       "title    3 Ninjas: High Noon At Mega Mountain (1998)  39 Steps, The (1935)  \\\n",
       "user_id                                                                      \n",
       "0                                                NaN                   NaN   \n",
       "1                                                NaN                   NaN   \n",
       "2                                                1.0                   NaN   \n",
       "3                                                NaN                   NaN   \n",
       "4                                                NaN                   NaN   \n",
       "\n",
       "title    ...  Yankee Zulu (1994)  Year of the Horse (1997)  \\\n",
       "user_id  ...                                                 \n",
       "0        ...                 NaN                       NaN   \n",
       "1        ...                 NaN                       NaN   \n",
       "2        ...                 NaN                       NaN   \n",
       "3        ...                 NaN                       NaN   \n",
       "4        ...                 NaN                       NaN   \n",
       "\n",
       "title    You So Crazy (1994)  Young Frankenstein (1974)  Young Guns (1988)  \\\n",
       "user_id                                                                      \n",
       "0                        NaN                        NaN                NaN   \n",
       "1                        NaN                        5.0                3.0   \n",
       "2                        NaN                        NaN                NaN   \n",
       "3                        NaN                        NaN                NaN   \n",
       "4                        NaN                        NaN                NaN   \n",
       "\n",
       "title    Young Guns II (1990)  Young Poisoner's Handbook, The (1995)  \\\n",
       "user_id                                                                \n",
       "0                         NaN                                    NaN   \n",
       "1                         NaN                                    NaN   \n",
       "2                         NaN                                    NaN   \n",
       "3                         NaN                                    NaN   \n",
       "4                         NaN                                    NaN   \n",
       "\n",
       "title    Zeus and Roxanne (1997)  unknown  Á köldum klaka (Cold Fever) (1994)  \n",
       "user_id                                                                        \n",
       "0                            NaN      NaN                                 NaN  \n",
       "1                            NaN      4.0                                 NaN  \n",
       "2                            NaN      NaN                                 NaN  \n",
       "3                            NaN      NaN                                 NaN  \n",
       "4                            NaN      NaN                                 NaN  \n",
       "\n",
       "[5 rows x 1664 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "moviemat = df.pivot_table(index='user_id',columns='title',values='rating')\n",
    "moviemat.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's normal for there to be lots of NaN values, as not everyone would have seen most movies."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Checking out the Most rated movie:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rating</th>\n",
       "      <th>num of ratings</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Star Wars (1977)</th>\n",
       "      <td>4.359589</td>\n",
       "      <td>584</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Contact (1997)</th>\n",
       "      <td>3.803536</td>\n",
       "      <td>509</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Fargo (1996)</th>\n",
       "      <td>4.155512</td>\n",
       "      <td>508</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Return of the Jedi (1983)</th>\n",
       "      <td>4.007890</td>\n",
       "      <td>507</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Liar Liar (1997)</th>\n",
       "      <td>3.156701</td>\n",
       "      <td>485</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>English Patient, The (1996)</th>\n",
       "      <td>3.656965</td>\n",
       "      <td>481</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Scream (1996)</th>\n",
       "      <td>3.441423</td>\n",
       "      <td>478</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Toy Story (1995)</th>\n",
       "      <td>3.878319</td>\n",
       "      <td>452</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Air Force One (1997)</th>\n",
       "      <td>3.631090</td>\n",
       "      <td>431</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Independence Day (ID4) (1996)</th>\n",
       "      <td>3.438228</td>\n",
       "      <td>429</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                 rating  num of ratings\n",
       "title                                                  \n",
       "Star Wars (1977)               4.359589             584\n",
       "Contact (1997)                 3.803536             509\n",
       "Fargo (1996)                   4.155512             508\n",
       "Return of the Jedi (1983)      4.007890             507\n",
       "Liar Liar (1997)               3.156701             485\n",
       "English Patient, The (1996)    3.656965             481\n",
       "Scream (1996)                  3.441423             478\n",
       "Toy Story (1995)               3.878319             452\n",
       "Air Force One (1997)           3.631090             431\n",
       "Independence Day (ID4) (1996)  3.438228             429"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings.sort_values('num of ratings',ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's choose two movies to focus on: starwars, a sci-fi movie. And Dumb and Dumber, a comedy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>rating</th>\n",
       "      <th>num of ratings</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>'Til There Was You (1997)</th>\n",
       "      <td>2.333333</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1-900 (1994)</th>\n",
       "      <td>2.600000</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101 Dalmatians (1996)</th>\n",
       "      <td>2.908257</td>\n",
       "      <td>109</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12 Angry Men (1957)</th>\n",
       "      <td>4.344000</td>\n",
       "      <td>125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>187 (1997)</th>\n",
       "      <td>3.024390</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             rating  num of ratings\n",
       "title                                              \n",
       "'Til There Was You (1997)  2.333333               9\n",
       "1-900 (1994)               2.600000               5\n",
       "101 Dalmatians (1996)      2.908257             109\n",
       "12 Angry Men (1957)        4.344000             125\n",
       "187 (1997)                 3.024390              41"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ratings.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's grab the user ratings for those two movies:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "user_id\n",
       "0    5.0\n",
       "1    5.0\n",
       "2    5.0\n",
       "3    NaN\n",
       "4    5.0\n",
       "Name: Star Wars (1977), dtype: float64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "starwars_user_ratings = moviemat['Star Wars (1977)']\n",
    "dumb_user_ratings = moviemat['Liar Liar (1997)']\n",
    "starwars_user_ratings.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can then use corrwith() method to get correlations between two pandas series:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/sajal/.pyenv/versions/3.10.0/envs/ds_portfolio/lib/python3.10/site-packages/numpy/lib/function_base.py:2821: RuntimeWarning: Degrees of freedom <= 0 for slice\n",
      "  c = cov(x, y, rowvar, dtype=dtype)\n",
      "/Users/sajal/.pyenv/versions/3.10.0/envs/ds_portfolio/lib/python3.10/site-packages/numpy/lib/function_base.py:2680: RuntimeWarning: divide by zero encountered in true_divide\n",
      "  c *= np.true_divide(1, fact)\n"
     ]
    }
   ],
   "source": [
    "similar_to_starwars = moviemat.corrwith(starwars_user_ratings)\n",
    "similar_to_liarliar = moviemat.corrwith(dumb_user_ratings)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's clean the data by removing NaN values and using a DataFrame instead of a series:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Correlation</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>'Til There Was You (1997)</th>\n",
       "      <td>0.872872</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1-900 (1994)</th>\n",
       "      <td>-0.645497</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101 Dalmatians (1996)</th>\n",
       "      <td>0.211132</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12 Angry Men (1957)</th>\n",
       "      <td>0.184289</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>187 (1997)</th>\n",
       "      <td>0.027398</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           Correlation\n",
       "title                                 \n",
       "'Til There Was You (1997)     0.872872\n",
       "1-900 (1994)                 -0.645497\n",
       "101 Dalmatians (1996)         0.211132\n",
       "12 Angry Men (1957)           0.184289\n",
       "187 (1997)                    0.027398"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_starwars = pd.DataFrame(similar_to_starwars,columns=['Correlation'])\n",
    "corr_starwars.dropna(inplace=True)\n",
    "corr_starwars.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we sort the dataframe by correlation, we should get the most similar movies, however there will be some results that don't really make sense. \n",
    "\n",
    "This is because there are a lot of movies only watched once by users who also watched star wars (it was the most popular movie). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Correlation</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Hollow Reed (1996)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Commandments (1997)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Cosi (1996)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>No Escape (1994)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Stripes (1981)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Star Wars (1977)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Man of the Year (1995)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Beans of Egypt, Maine, The (1994)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Old Lady Who Walked in the Sea, The (Vieille qui marchait dans la mer, La) (1991)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Outlaw, The (1943)</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    Correlation\n",
       "title                                                          \n",
       "Hollow Reed (1996)                                          1.0\n",
       "Commandments (1997)                                         1.0\n",
       "Cosi (1996)                                                 1.0\n",
       "No Escape (1994)                                            1.0\n",
       "Stripes (1981)                                              1.0\n",
       "Star Wars (1977)                                            1.0\n",
       "Man of the Year (1995)                                      1.0\n",
       "Beans of Egypt, Maine, The (1994)                           1.0\n",
       "Old Lady Who Walked in the Sea, The (Vieille qu...          1.0\n",
       "Outlaw, The (1943)                                          1.0"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_starwars.sort_values('Correlation',ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's fix this by filtering out movies that have less than 100 reviews (this value was chosen based off the histogram from earlier)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Correlation</th>\n",
       "      <th>num of ratings</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>'Til There Was You (1997)</th>\n",
       "      <td>0.872872</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1-900 (1994)</th>\n",
       "      <td>-0.645497</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101 Dalmatians (1996)</th>\n",
       "      <td>0.211132</td>\n",
       "      <td>109</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12 Angry Men (1957)</th>\n",
       "      <td>0.184289</td>\n",
       "      <td>125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>187 (1997)</th>\n",
       "      <td>0.027398</td>\n",
       "      <td>41</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           Correlation  num of ratings\n",
       "title                                                 \n",
       "'Til There Was You (1997)     0.872872               9\n",
       "1-900 (1994)                 -0.645497               5\n",
       "101 Dalmatians (1996)         0.211132             109\n",
       "12 Angry Men (1957)           0.184289             125\n",
       "187 (1997)                    0.027398              41"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_starwars = corr_starwars.join(ratings['num of ratings'])\n",
    "corr_starwars.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now sort the values and notice how the titles make a lot more sense:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Correlation</th>\n",
       "      <th>num of ratings</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Star Wars (1977)</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>584</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Empire Strikes Back, The (1980)</th>\n",
       "      <td>0.748353</td>\n",
       "      <td>368</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Return of the Jedi (1983)</th>\n",
       "      <td>0.672556</td>\n",
       "      <td>507</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Raiders of the Lost Ark (1981)</th>\n",
       "      <td>0.536117</td>\n",
       "      <td>420</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Austin Powers: International Man of Mystery (1997)</th>\n",
       "      <td>0.377433</td>\n",
       "      <td>130</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    Correlation  \\\n",
       "title                                                             \n",
       "Star Wars (1977)                                       1.000000   \n",
       "Empire Strikes Back, The (1980)                        0.748353   \n",
       "Return of the Jedi (1983)                              0.672556   \n",
       "Raiders of the Lost Ark (1981)                         0.536117   \n",
       "Austin Powers: International Man of Mystery (1997)     0.377433   \n",
       "\n",
       "                                                    num of ratings  \n",
       "title                                                               \n",
       "Star Wars (1977)                                               584  \n",
       "Empire Strikes Back, The (1980)                                368  \n",
       "Return of the Jedi (1983)                                      507  \n",
       "Raiders of the Lost Ark (1981)                                 420  \n",
       "Austin Powers: International Man of Mystery (1997)             130  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_starwars[corr_starwars['num of ratings']>100].sort_values('Correlation',ascending=False).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now the same for Liar Liar:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Correlation</th>\n",
       "      <th>num of ratings</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>title</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Liar Liar (1997)</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>485</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Batman Forever (1995)</th>\n",
       "      <td>0.516968</td>\n",
       "      <td>114</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mask, The (1994)</th>\n",
       "      <td>0.484650</td>\n",
       "      <td>129</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Down Periscope (1996)</th>\n",
       "      <td>0.472681</td>\n",
       "      <td>101</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Con Air (1997)</th>\n",
       "      <td>0.469828</td>\n",
       "      <td>137</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       Correlation  num of ratings\n",
       "title                                             \n",
       "Liar Liar (1997)          1.000000             485\n",
       "Batman Forever (1995)     0.516968             114\n",
       "Mask, The (1994)          0.484650             129\n",
       "Down Periscope (1996)     0.472681             101\n",
       "Con Air (1997)            0.469828             137"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "corr_liarliar = pd.DataFrame(similar_to_liarliar,columns=['Correlation'])\n",
    "corr_liarliar.dropna(inplace=True)\n",
    "corr_liarliar = corr_liarliar.join(ratings['num of ratings'])\n",
    "corr_liarliar[corr_liarliar['num of ratings']>100].sort_values('Correlation',ascending=False).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And we're done!\n",
    "\n",
    "It looks like our results make sense. Even though the system is far from state of the art, it still recommended the other Star Warss movies (and another George Lucas film) for Star Wars; while it looks like there's some scope for more improvement in Liar Liar (although we do get another Jim Carrey movie as a recommendation too."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
